Total: 18
Sign language is the primary mode of communication in the Deaf and Hard-of-Hearing (DHH) communities. Unfortunately, sign language is not as well understood among the non-signing hearing population leading to limited access and services to the DHH community, and also acts as a barrier between non-signing and DHH people. In my Ph.D. thesis, I am working on improving the sequence modeling for sign language translation and understanding by considering different types of sequence models, various input features, and by understanding the semantic relation between the words and the signs. Currently, my research focuses on a popular publicly available German Sign Language dataset.
Clustering unleashes the power of scRNA-seq through identification of appropriate cell groups. It is considered a pre-requisite to performing differential expression analysis, followed by functional profiling to identify potential biomarkers from scRNA-seq data. Most existing clustering methods either integrate cluster validity indices or need user assistance to identify clusters of arbitrary shape. We develop two clustering methods 1) UIFDBC to identify clusters of arbitrary shapes, 2) UIPBC to cluster scRNA-seq data. Neither method integrates a cluster validity index nor takes any user input. However, specialised approaches are used to benchmark the parameters. Both approaches outperform state-of-the-art methods.
We are at a turning point to address the unprecedented challenges we are facing in healthcare systems. With the aging of the population and increasing health disparities in rural areas, healthcare needs assistance from technologies to provide quality care for these populations. In collaboration with clinicians, we seek to meet this need by creating data-driven methods that provide interpretable healthcare models from ubiquitous ambient and wearable sensor data. My doctoral research goal is to introduce novel ways to help clinicians understand patients' health status by developing new visualization tools and interpretable models that analyze human health and behavior from sensor data.
Neural Networks (NNs) are popular machine learning models which have found successful application in many different domains across computer science. However, it is hard to provide any formal guarantee on the behaviour of neural networks and therefore their reliability is still in doubt, especially concerning their deployment in safety and security-critical applications. Verification emerged as a promising solution to address some of these problems. In the following, I will present some of my recent efforts in verifying NNs.
My research aims to enable spatiotemporal inference in mobile robot perception systems. Specifically, the proposed thesis presents learning-based approaches to the tasks of behavior prediction and occlusion inference that explicitly account for the associated aleatoric and epistemic uncertainty.
My research activity focuses on the integration of acting, learning and planning. The main objective is to build a system that is capable to learn how to plan and act in an unknown, dynamic and complex environment. The only knowledge the agent has about the environment is provided by a set of sensor observations which returns continuous measures on the environment. On the learning side, I’m interested in developing algorithms that allow an artificial agent to learn an abstract model of the dynamics of the environment (either an explicit model like a deterministic finite state machine or a model described in a language to express planning domains). The type of abstract model is specified by means of discrete state variables rather than continuous variables representing agent observations. In addition to learning the abstract model, I’m interested in learning probabilistic (generative) models that connects the abstract model with the perceptions of the agents. On the acting and planning side, the artificial agent does not rely on a prior set of execution traces, it rather decides online how to act by means of state-of-art planners. With its own experience, it enriches the planner knowledge, as well as the learned model of the environment. On the learning part, the agent applies techniques for dynamic probabilistic clustering of perceptions in a set of abstract states, neural network for learning transition models, and inductive reasoning for learning action model descriptions. Notice that this is different from Reinforcement Learning where the focus is to learn a policy for achieving a goal, we are interested in learning an abstract model of the environment. We do not have a reward function that encodes the goal to be reached. Indeed in this work an agent does not necessarily need to reach a goal.
The availability of large training data has led to the development of sophisticated deep learning algorithms to achieve state-of-the-art performance on various tasks and several applications have been benefited immensely. Despite the unparalleled success, the performance of deep learning algorithms depends significantly on the training data distribution. An imbalance in training data distribution affects the performance of deep models. Our research focuses on designing and developing solutions for different real-world problems, specifically related to facial analytic tasks, with imbalanced data distribution. These problems include injured face recognition, fake image detection, and estimation and mitigation of bias in model prediction.
Depression has increased at alarming rates in the worldwide population. One alternative to finding depressed individuals is using social media data to train machine learning (ML) models to identify depressed cases automatically. Previous works have already relied on ML to solve this task with reasonably good F-measure scores. Still, several limitations prevent the full potential of these models. In this work, we show that the depression identification task through social media is better modeled as a Multiple Instance Learning (MIL) problem that can exploit the temporal dependencies between posts.
The rapid growth of large scale event datasets with timestamps has given rise to the dynamically evolving multi-relational knowledge graphs. Temporal reasoning over such data brings on many challenges and is still not well understood. Most real-world knowledge graphs are characterized by a long-tail relation frequency distribution where a significant fraction of relations occurs only a handful of times. This observation has given rise to the recent interest in low-shot learning methods that are able to generalize from only a few examples. The existing approaches, however, are tailored to static knowledge graphs and not easily generalized to temporal settings, where data scarcity poses even bigger problems, due to the occurrence of new, previously unseen relations. The goal of my doctoral research is to introduce new approaches for learning meaningful representation that capture the dynamics of temporal knowledge graphs while tackling various existing challenges such as data scarcity.
Two fundamental challenges in program synthesis, i.e. learning programs from specifications, are (1) program correctness and (2) search efficiency. We claim logical constraints can address both: (1) by expressing strong requirements on solutions and (2) due to being effective at eliminating non-solutions. When learning from examples, a hypothesis failing on an example means that (a class of) related programs fail as well. We encode these classes into constraints, thereby pruning away many a failing hypothesis. We are expanding this method with failure explanation: identify failing sub-programs the related programs of which can be eliminated as well. In addition to reasoning about examples, programming involves ensuring general properties are not violated. Inspired by the synthesis of functional programs, we intend to encode correctness properties as well as runtime complexity bounds into constraints.
Social robots deployed in the therapy of autism is a promising and important research domain. Recently, an increasing amount of work is being conducted utilizing a social robot as a mediator between a therapist and a child with autism. Being able to evaluate how engaged a child is both offline and in real-time would improve the quality of the provided robot-assisted intervention and also provide objective metrics for later analysis by the therapist. The state-of-the-art engagement recognition is challenged by the diverse styles of expressing engagement by this vulnerable population group. To this end, this PhD project aims to explore how transfer learning can improve the recognition accuracy of children's engagement with the robot or another human. We will utilize four publicly available multi-modal datasets to discover a suitable feature representation of engagement during various types of activities with the robot.
Interactive Task Learning (ITL) is an approach to teaching robots new tasks through language and demonstration. It relies on the fact that people have experience teaching each other. However, this can be challenging if the human instructor does not have an accurate mental model of a robot. This mental model consists of the robot’s knowledge, capabilities, shortcomings, goals, and intentions. The research question that I investigate is “How can the robot help the human build a better mental model of the robot?”
AI for social good (AI4SG) is a research theme that aims to use and advance AI to improve the well-being of society. My work on AI4SG builds a two-way bridge between the research world and the real world. Using my unique experience in food waste and security, I propose applied AI4SG research that directly addresses real-world challenges which have received little attention from the community. Drawing from my experience in various AI4SG application domains, I propose bandit data-driven optimization, the first iterative prediction-prescription framework and a no-regret algorithm PROOF. I will apply PROOF back to my applied work on AI4SG, thereby closing the loop in a single framework.
The use of machine learning components in safety-critical systems creates reliability concerns. My thesis focuses on developing algorithms to address these concerns. Because the assurance of a safety-critical system generally requires multiple types of validation, my research takes three directions: safe deep learning algorithms, formal verification of neural networks, and adaptive testing methods.
Peer review is the backbone of academia. The rapid growth of the number of submissions to leading publication venues has identified a need for automation of some parts of the peer-review pipeline and nowadays human referees are required to interact with various interfaces and technologies in this process. However, there exists evidence that if such interactions are not carefully designed, they can exacerbate various problems related to fairness and efficiency of the process. In my research, I aim to design a Human-AI collaboration pipeline in peer review to mitigate these issues and ensure that science progresses in a fair, equitable, and efficient manner.
We study the application of multi-agent reinforcement learning for game-theoretical problems. In particular, we are interested in coalition formation problems and their variants such as hedonic coalition formation games (also called hedonic games), matching (a common type of hedonic game), and coalition formation for task allocation. We consider decentralized multi-agent systems where autonomous agents inhabit an environment without any prior knowledge of other agents or the system. We also consider spatial formulations of these problems. Most of the literature for coalition formation problems does not consider these formulations of the problems because it increases computational complexity significantly. We propose novel decentralized heuristic learning and multi-agent reinforcement learning (MARL) approaches to train agents, and we use game-theoretic evaluation criteria such as optimality, stability, and indices like Shapley value.
This thesis focuses attention on achieving Distributed Situation Awareness (DSA) with minimal resources (energy, processing cost, etc.) using small low-capacity agents (e.g., UAVs) coordinated in a decentralised fashion while conducting searching activity. This is in contrast to the existing works involving convoluted communication and information processing.
A paradigm shift towards human-centered intelligent gaming systems is gradually setting in. Such intelligent gaming systems with embedded machine learning algorithms would explain player motivations, help design more personalized single and collaborative player experiences, transfer and generalize the learning from game to game. The multi-modal user behavior trajectories, both in-game and across various platforms, incorporate heterogeneous information and graph structures. These gaming modalities range from text, audios, video demos, activity replays, and social networks to psychological questionnaires. Identifying decision-making patterns and strategies by observing in-game behavior actions and mining heterogeneous sources could construct a more holistic representation of the gaming community. Human priors publicly available on the World Wide Web would inspire the modeling for human-like non-player characters, adaptive recommendation systems, automatic game design, testing, and human-AI collaborations. My doctoral research goal is to mine, represent, and learn from human priors existing in the interactive entertainment community's heterogeneous sources and introduce ways to model single and multi-agent interactive behavior patterns.